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INVESTIGATION OF MODELS FOR LARGE-SCALE 




METEOROLOGICAL PREDICTION EXPERIMENTS 


Introduction 

The principal activities of The City College group during the 
past year were all related to the evaluation of monthly mean forecasts 
and climate simulations generated at the Goddard Institute for 
Space Studies (GISS) with the global "climate model" (Hansen, et al.lP79). 
This work was conducted mainly at GISS, and in close collaboration 
with the GISS atmospheric modeling team. All computations, including 
plots, were carried out at GISS, which aloO provided office space for 
the principal investigator and 2 or 3 graduate student research assistants. 

Graduate assistants employed on the project during the past year 
included Ronald Filadelfo (who completed his master's program at The 
City College and resigned August 31, 1979 to pursue a Ph.D. program in 
oceanography at SUNY-Stony Brook), Zaphiris Christidis, and Michael Dennis 
who joined the group in July 1979. The City College project has been 
ably assisted also by two GISS employees, Jesus J. Notario (who resigned 
in October 1979) and Robert Klugman. 

Most of the forecast and climate simulation experiments analyzed 
were conducted with a "medium" mesh (8° of latitude by 10° of longitude) 
7-layer version of the climate model, referred to as MX500M7. However, 
experiments have also been run with 9- and 12 • layer versions and 
with a "fine" mesh (4° of latitude by 5° of longitude) . In the course 
of the model development, various modifications were introduced which 

delayed the forecast verifications for several months, and which also 
rendered many early experiments obsolete. 
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The evaluation program has concentrated on two kinds of outputs: 

(a) monthly mean forecasts for the months of October 1976 through 
February 1977, generated from global initial conditions for the first 
day of each month provided by the National Meteorological Center (NMC) 
and the National Center for Atmospheric Research (NCAR) , and (5) a "model 
climatology" generated by running the model for 5 simulated years from 
one set of initial conditions (I December 1976) and then averaging the 
5 outputs for each of the 12 calendar months. The forecasts were 
evaluated against the corresponding monthly mean NMC/NCAR data for the 
5 specific months above, while the model climatology was compared with 
an "observed climatology" provided by NCAR and based on data archived 
at the National Climate Center (NCC) in Asheville, N.C. 

This has been primarily a synoptic evaluation , with attention focussed 
mainly on three field: 850 mb temperatures (T8), 500 mb geopotential 
heights (Z5) , and sea-level pressures (SLP) . The results are presented 
in map form, and also numerically in terms of various measures of 
agreement, including root-mean-square errors, gradient (SL) skill scores, 
and correlation coefficients between forecast and observed fields, or 
between model and observed climatologies. The observed and model -generated 
synoptic fields were also subjected to spherical harmonic analysis 
and compared in terms of the principal modes. 

One paper was published in 1979 (Spar, J. and R. Lutz, 1979: 
Simulations of the monthly mean atmosphere for February 1976 with the 
GISS model. Monthly Weather Review, 107, 181-192), and the following 
technical reports, as well, as the semi-annual status report, were 
distributed: 
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Spar, J., 1979: Interim report on forecasting experiments 

with a climate model. 4 pp. June 1979. 

Spar, J. and R. Klugman, 1979:’ Note on decay of predictability 
in forecast experiments with the GISS climate model. 

3 pp June 1979. 

Spar, J. , R. Klugman and J. J. Not^rio, 1979: Effects of 

horizontal and vertical resolution in climate model 
forecast experiments. 8 pp. July 1979. 

Christidis, 1 . and J. Spar, 1979: Spherical harmonic 

analysis for verification of a global atmospheric 
model. 9 pp. August 1979 

An invited paper on "Prediction Experiments with a Coarse-Mesh 
Global Model" was presented by J. Spar at the Climate Diagnostics 
Workshop in Madison, Wis (16-18 October 1979), and will be published 
in the proceedings of the Workshop. 

A master’s thesis by R. Filadelfo ^'The Effect of Sea Surface 
Temperature Anomalies on Monthly Mean Simulations with a Coarse Mesh 
Global General Circulation Model") was accepted by The City College 
in October 1979. 


Monthly Mean Forecast Experiments with the 
Climate Model 

The 8“ x 10® climate model was run using global NMC/NCAR initial 
conditions for 00 GMT on the first day of each of 5 months, October 1976 
through February 1977, a period of severe weather over the eastern 
United States. Initially a 9- level version of the model (No. 252) was 
used for these experiments, and later the forecasts were repeated with 
a 7- level version (No. 444). As reported in the Semi-Annual Status 
Report of the project (June 1979), neither set of monthly mean forecasts 
exhibited skill equal to that of climatology, indicating that these 
versions of the model were neither predicting climatology nor correctly 
predicting the anomalous departures from climatology of the monthly 
mean fields over the globe or over the Northern Hemisphere. 


- 3 - 


X 


The low skill of the monthly mean forecasts is, of course, 
related to the rapid decay of predictability. This is illustrated 
by the results of an evaluation of the forecast history generated by 
model No. 444 in which the 5 forecast runs were averaged over periods 
of the first 5, 10, and 15 days, as well as the whole month. The SI 
skill scores and rms errors over the Northern Hemisphere were computed 
for each averaging period, and averages of these values for the 
5-month experiment are shown in figure 1. It is apparent that most 
of the forecast error (about 90?;) is generated in the first 15 days 
of the month. 

The monthly mean forecasts were later repeated with model number 
MX500M7 (another 8®xl0® 7- level version) with similar results, as 

shown by the forecast statistics for the Northern Hemisphere in Table 1. 
The discrepancies between the predicted and observed monthly mean 
synoptic fields are illustrated for the month of December 1976 in 
figures 2-4, vtfith the forecast fields at the top and the observed 
fields at the bottom. While the forecast fields of T8 and Z5 are 
realistic in appearance, there are obvious errors in the simulations 
of the large-scale temperature and contour patterns. The discrepancies 
between the forecast and observed SLP fields are, of course, even 
more apparent . 

The forecast errors do not appear to be highly dependent on the 
grid size employed. This is indicated by a comparative run for December 
1976 that was carried out with the same 7- layer model (No. 444), but 
with two different grids: 8®xl0® and 4®x5° . As shown in Table 2 

below, the rms errors, which are not scale -dependent , are very similar 
for the two grids. The SI skill scores, which are seen to be highly 
scale dependent, are actually higher (i.e., worse) for the finer grid. 
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Averaaina Period (Days ) 



:OPOTENTIAL° HEIGHT 








Table 1. Forecast statistics for the Northern Hemisphere with 

model MX500M7 (M) , October 1976-February 1977, compared 
witli "forecasts" of climatology (C) 


October 1976 


T8-M 


T8-C 


Z5-M 


Z5-C 


SLP-M 


SLP-C 


rms error 


3.2°C 


2.2°C 


62m 


39m 


6 . Omb 


3 . Omb 


SI score 


65 


49 


63 


47 


99 


65 


November 1976 


rms error 


3.8“C 


2.1°C 


72m 


40m 


7.1mb 


2 .9mb 


SI score 


66 


47 


66 


42 


90 


53 


December 1976 


rms error 

3.4®C 

QQQIII 

71m 

43m 

5 . 8mb 

3 . 5mb 

SI score 

62 

39 

63 

43 

84 

51 




January 1977 


rms error 


4.3°C 


2.7°C 


91m 


73m 


6 . 8mb 


5 . Omb 


SI score 


61 


51 


66 


57 


8 3 


64 


Febyuary 1977 


rms error 


SI score 


4.2“C 


68 


2.3°C 


43 


80m 


60 


5 5m 


48 


7 . 5mb 


85 


4 . 4mb 


62 



















































but the difference is just about accounted for by the scale-dependence. 


I 

j 

s 

j 

The correlation coefficients, r, between the forecast and observed i 

deviations from climatology (i.e., between anomalies), are higher for \ 

i 

the finer grid, but the scale-dependence of this statistic has not j 

been determined. The statistics for a forecast of climatology on the 
two grids, which are also included in the table, indicate that the 
errors relative to climatology are similar on the two grids. 

Table 2 . Forecast statistics for December 1976, Northern Hemisphere, 
on 4°x5“ and 8°xl0® grids with model No. 444. Rms errors 
and SI scores for climatology are also shown for comparison. 



T8 

Z5 



4x5 


■ 

8x10 

4x5 


SLP 


8x10 


Model 444 


rms 

3.1^C 

3.2°C 

66m 

66m 

1 

5.6 mb 

6 . 3mb 

SI 

93 

54 

89 

63 

104 

87 

r 

+ 0.36 

+ 0.23 

+ 0.14 

+ 0.02 

+ 0.41 

+ 0.30 


Climatology 


rms 


1.9°C 


75 


1.9"C 


39 


43m 


75 


43m 


43 


3.5 mb 


86 


3.5 mb 


51 

































The fact that the model does not generate a monthly mean forecast 


that is better than (or even as good as) climatology, suggests that 
the model’s "climatolojiy" may be quite different that that of the 
real world. If this is so, then the forecast state for a given month 
would obviously correlate poorly with the observed. However, it is 
possible that, even if the two climatologies are different, the response 
of the model to changes in initial conditions might still resemble 
that of nature. As a test of this hypothesis, correlation coefficients, 
r*, were computed between the observed monthly anomaly, i.e. the 
observed deviation from the real climatology, and the model * s forecast 
anomaly , i.e. the deviation of the forecast from the model * s monthly 
climatology. This differs from the correlation coefficient, r (as, e.g,, 
in Table 2) , for which both observed and forecast anomalies were 
computed relative to the same real climatology. (The calculation of 
r* necessitated the generation, .of a model climatology, which is discussed 
in further detail later in this report.) 

Table 3 shows the results of the verification of the 5 monthly 

mean forecasts generated by model MX500M7 in terms of the two 

correlation coefficients, r* and r. If, despite any differences between 

the two climatologies, the model correctly simulated the response of 

the atmosphere to the given initial conditions, r* should be higher 

than r. However, this is seen to be true in only 7 out of the 15 trials, 

and the mean values of r* and r are both almost equally low. Thus, the 

hypothesis that model correctly simulates response to initial 

conditions does not appear to be supported by this experiment. (An 

alternative experiment, in which the model is initialized with data 

representing the same deviation from the model climatology as the actual 

deviation of the observed initial state from the real climatology, has 
not yet been carried out.) 
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Table 3. 


Correlation coefficient, r*, between model forecast 
anomaly (forecast - model climatology) and observed 
anomaly (observed - real climatology) compared with 
correlation coefficient, r, between forecast and 
observed anomalies relative to the real climatology. 
Northern Hemisphere Model MX500M7. 


Month 

Field 

r* 

r 

October 1976 

T8 

+ 0.08 

+ 0.56 

Z5 

+ 0.3S 

+ 0.30 

SLP 

♦0.20 

+ 0.39 

November 1976 

T8 

+ 0.05 

+ 0.20 

Z5 

+ 0.17 

-0.11 

SLP 

+ 0.03 

-0.21 

December 1976 

T8 

“0.05 

" f ■ ' 

+ 0.15 

Z5 

+ 0.14 

-0.03 

SLP 

+ 0.28 

+ 0.38 

January 1977 

T8 

+ 0.00 

+ 0.07 

Z5 

+ 0.34 

+ 0.17 

SLP 

+ 0.32 

+ 0.36 

February 1977 

T8 

+ 0.07 

+ 0.20 

Z5 

+ 0.14 

+ 0.12 

SLP 

+ 0.17 

+ 0.05 

Mean 


+ 0.16 

+ 0.17 














































The influence of SST anomalies On monthly and seasonal atmospheric 
states is considered to be an important one by many subjective and 
.statistical long-range weather forecasters. However, efforts to 
demonstrate such atmospheric forcing by anomalous surface boundary 
conditions through the use of dynamical models have generally yielded 
ambiguous results, mainly due to model deficiencies (low forecast skill) 
and the dubious quality of the SST data. Nevertheless, an effort was 
made to test the response of the GISS climate model to SST anomalies by 
computing two parallel forecasts from identical initial conditions, but 
with the usual monthly climatological SST's used in one run (the "control" 
forecast) and the "observed" monthly mean SST's used in the other (the 
"anomaly" forecast) as ocean surface boundary conditions. 

Daily global SST data, derived from NOA/V satellite scanning 
radiometer measurements (Brower, et al., 1976), are available on tape from 
NMC on two 256 x 256 polar stereographic grids. These values were 
averaged over each month, the monthly mean stereographic values were 
interpolated to a 1® latitude-longitude grid, and the spherical grid data 
were then averaged over 8°xl0° grid boxes of the model to obtain a 
suitably defined observed monthly mean SST field for each forecast. The 
use of such observed SST's in a forecast, of course, implies the hope 
that in some future interactive air-sea model it may be possible to 
predict accurately the SST field. 

Unfortunately, the satellite-derived SST's are now known to contain 
some rather serious errors (Barnett et al., 1979) which affect even the 
monthly averages, and as a result the validity of this experiment i;s 
rather questionable in terms of the influence of SST's on forecast skill. 
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Nevertheless, the experiment does reveal something about the response 
of the model to SST anomalies, even though the latter may be spurious. 

The global field of SST anomalies based on the satellite data for 

December 1976 is shown in figure 5, and the corresponding difference 

maps for the anomaly-minus-control forecasts of T8, Z5, and SLP are 

shown in figures 6-8. The dominant SST anomalies are warm water off 

the east coasts of North America and Asia near latitude 45“N, and cold 

water in the tropical North Pacific Ocean. Over the warm pools the model 

produces higlier 8 50 mb temperatures and 500 mb heights and lower sea- level 

pressures, but there is apparently little response to the cold water 

in the tropics. Large differences between the anomaly and control forecasts 

are found north of latitude 45® N, However, the "noise level" of the 

model atmosphere in high latitudes (based on the standard deviations 

for each month of the 5 year model climatology run described below) was 

found to be so large that no statistical significance could be attached 

•» 

to the anomaly -control differences in that region. Where the "noise 
level" is low (tlie tropics) , the "signal" (anomaly - control difference) 
is also small. 

(Further details of the SST anomaly experiment may be found in an 
M.A. thesis by R. Filadelfo: "The Effect of Sea-Surface Temperature 

Anomalies on Monthly Mean Simulations with a Coarse-Mesh Global General 
Circulation Model".) 
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The Model Climatology 


The I'esults of the model forecast experiments clearly indicated 
a need to learn more about the climatology which the model generates. 

In fact, the evaluation of the model climatology now takes precedence 
over the admittedly premature monthly mean forecast experiments. 

Before any more forecasts are attempted, it must be determined whether 

I 

or not the model produces a realistic climatology. To this end, the 
- MX500M7 model was initialized with NMC/NCAR data for 1 December 1976, and 
was run for a period of 5 simulated years. The 12 montlily average fields 
derived from the 5 year simulation are considered to represent the 
"model climatology". 

To evaluate the model climatology, it was compared with a climatological 
data set available on tape from NCAR and based on data archived at NCC. 

A numerical comparison of the two climatologies is shown in Table 4, in 
which the bias (algebraic mean difference between model and observed 
climatology) as well as rms differences and SI scores are tabulated for 
5 months, October - February, for the Northern Hemisphere. 

Comparing Table 4 with Table 1, it appears that the discrepancy 
between the two synoptic climatologies is quantitatively similar to the 
discrepancy between the forecast and observed monthly mean fields, which 
does suggest that the forecast errors are associated with "errors" in 
the model climatology. 

The differences between the two climatologies are illustrated in 
figures 9-11 showing the model climatology (top) and the observed 
climatology (bottom) for the month of December. While the model climatology 
is generally rather realistic in appearance, there are some notable 
differences between the two climatologies, particularly in the SLP field 
and in the sub-tropical land temperatures in the summer hemisphere. 
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Table 4. Five-year "model climatology" versus observed (NCAR) 
climatology difference statistics for the Northern 
Hemisphere . 


Month 

Field 

Bias . 

Rms 

SI 

October 

T8 

-n.2°c 

3.8“C 

7 2 

Z5 

-29m 

51m 

5 7 

SLP 

- 1 . 2mb 

6 . 9mb 


November 

T8 

-0.5°C 

4 .n°c 

70 

Z5 

-4 0m 

film 

5 2 

SLP 

- 1 . 6mb 

6 . 2mb 

89 

December 

T8 

-0.5“C 

CO 

O 

O 

58 

Z5 

-44m 

fifim 

54 

SLP 

- 2 . Imb 

6 . Imb 

87 

January 

T8 

-0.5“C 

3.8°C 

60 

Z5 

- 53m 

7 7m 

54 

SLP 

- 2 . 4mb 

fi . Imb 

89 

February 

T8 

-0.6°C 

4 .1°C 

61 

Z5 

- 5 2m 

7fim 

5 3 

SLP 

- 2 . Smb 

fi . 2mb 

91 


































which are too warm in the model simulation. 

From the model climatology run it is possible to determine the 
interannual variances of the model atmosphere for each month. These are 
illustrated for the month of December in figures 12-14, showing global 
maps of standard deviations of T8, Z5, and SLP. The magnitudes of the 
model standard deviations are similar to those of the real atmosphere 
(e.g., Crutcher and Meseive. 1970), with large values in high latitudes 
and low values in the tropics. However, tlie Arctic and Antarctic standard 
deviations (which are indicative of the model "noise level") appear to be 
extremely high. 
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Spherical Harmonic Analysis 

A comparison of model -generated and observed synoptic fields was 
also carried out in terms of spherical harmonic functions. (See, e.g., 
Chapman and Bartels, 1962.) In particular, the model -generated and 
observed monthly climatological fields of T8, Z5 and SLP were subjected 
to spherical liarmonic analysis, and the coefficients (Cn,m and Sn,m) 
of the even and odd normalized spherical harmonic functions for each 
month were evaluated up to degree, and order, m, equal to 18. 

Amplitudes (An,m^ = Cn,m2 * Sn,m2) and phase angles (Bn,m = tan‘1 
Sn,m/Cn,m) were computed for each harmonic, and the fields were compared 
in terms of the principal modes. The analysis was done using both 
global data and northern hemisphere data alone, with the latter reflected 
into the Southern Hemisphere for computational purposes. 

f 

Figue 15 illustrates the annual cycle of amplitudes and phases for 
the Northern Hemisphere of the model (dashed) and observed (solid) 
climatological fields of T8, Z5, and SLP for one component: n = 4, 
m = 2. The 4,2 harmonic has 2 nodal parallels (n-m = 2), one in each 
hemisphere, and a longitudinal wave number (m) of 2, corresponding to 
the distribution of oceans and continents in the Nortliern Hemisphere. 
(However, this is not the dominant harmonic. For TS and Z5, the dominant 
mode is the zonal harmonic 2,0, representing the difference between the 
Equator and the poles.) Although the annual cycle of the model for the 
4,2 harmonic does resemble tliat of the actual climatology, one notable 
discrepancy is due to the large amplitude which the model generates in 
T8 and SLP in summer, apparently as a consequence of excessive heating 
of the continents. (A diagnostic study by L. Druyan at GISS indicates 
that this may be caused by inadequate treatment of soil moisture.) 

Except for the transitional seasons, the monsoonal phase shifts of T8 
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and SLP are well-represented by the model, and the amplitude cycle for 
Z5 is quite realistic as well. 

The annual cycle of the global ZO harmonic, shown in figure 16, 
illustrates the generally good simulation .of the meridional profiles of 
T8 and Z5, and the poor representation of the SLP field. (However, the 
.2,0 harmonic is not as dominant a component of SLP as of T8 and Z5.) 

A comparison can be made between the model and actual climatologies 
by examining the leading harmonics of the two for any montli. In table 5 
are listed the first six harmonics (mode, amplitude, phase angle) of the 
model and NCAR climatologies for the month of December based on the 
global data and also on data from the Northern Hemisphere alone. (The 
mean value, represented by the 0,0 harmonic is not included in the list.) 

Based on the leading six harmonics for each field shown in Table 5 
it is again apparent tliat the model simulates the T8 and Z5 Climatological 
fields somewhat more successfully, than it does the SLP field. Among 
the global harmonics for T8, four of the six observed modes (2,0; 1,0; 

3,2; and 3,1) are found among the first six model harmonics, with 
approximately correct amplitudes and pliases , and the same is true of 
four of the six observed Z5 Iiarmonics (2,0; 6,0; 4,0; and 4,1). 

Four of the first six global SLP observed modes (1,0; 2,0; 4,0; and 2,1) 

are also found among the first six model harmonics, but tlie amplitudes 

of two of them (1,0 and 2,0) are not in good agreement. From the 

Northern Hemisphere harmonics in Table 5, similar conclusions can 

be drawn. The first three observed hemispheric T8 modes 

(2,0; 4,2; and 3,1) are found among the first four model harmonics, 

while five the six leading observed hemispheric Z5 modes 

(2,0; 6,0; 3,1; 5,3; 6,2) are found among the first six model harmonics, 
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Table 5. Six highest harmonics (mode, amplitude, phase angle) o£ the 

model and NCAR climatologies for December based on both global 
data and on Northern Hemisphere data only. 


T8 

model 


T8 

NCAR 


mode 

A 

B 


2,0 

42.5 


Globe 

1>0 

11.4 


6,0 

5.1 
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and in almost the same order. Although tlie first tliree observed 
hemispiieric SLP modes are represented among the first four model 
harmonics, tJie leading model harmonic (2,0) is not found among the 
first six observed modes for SLP. For most of the non-zonal harmonics 
(m>o) in Table 5 wJiich appear in botli the model and observed lists, 
the phase angles are in fairly good agreement. 
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